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(57) Abstract 

In a packet switch, which is intended for packets of constant length, and which, between switch ports, is divided into a 
number of nodes (12) and transmission links (14), where the nodes carry out space selection and the transmission links offer 
point-to-point transmission between the nodes, link grouping is carried out. More particularly, from multiple parallel physical 
linlcs coming into the switch, link groups are produced, each in the form of a logical link with a bandwidth that is the sum of the 
bandwidths of the physical links included in the link group, said logical link being restored to outgoing parallel physical links 
from the switch. One or more link protocols (GI, G2) are used according to which a label in the packet's header is made to de- 
scribe a route that holds together the grouped links through the entire switch, such that bits in the label describing the route over a 
certain transmission link are the same for packets belonging to the same link group. 
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A METHOD OF GR OUPING LTNKS IN A PACKET SWTTCH. 

5 

TECHNICAL RACKGROUNp. 

The present invention relates in general to link grouping 
in packet switches for packets with constant length, e.g., ATM 
switches {ATM = Asynchronous Transfer Mode) . Link grouping 

10 means that a logical link is created consisting of many 

parallel physical, links. The logical link offers a bandwidth 
that is the sun of that of the physical links. in this way 
can, for example, a logical link with a bandwidth of 622.08 
Mb/s be provided through link grouping of four physical 155,52 

15 Mb/s links. 

More particularly the invention relates to a method of 
grouping links in a packet switch for constant- length packets, 
the packet switch having a plurality of nodes and a plurality 
of transmission links for selectively connecting a plurality of 
input ports to a plurality of output ports via routes through 
the switch, wherein the nodes enable space selection and the 
transmission links interconnect the nodes. 

In an ATM-switch for wideband telecommunication there is a 
need for higher bandwidth than that which is physically 
25 available. Link grouping can offer good traffic characteristics 
and hardware flexibility and effectivity, for example smaller 
delay, fewer buffers, and the realization of different types of 
concentrators/multiplexers and switches in different speed 
classes while using the same base components. Blocking 
3 0 characteristics in a link coupled network, for example a 
Clos-network, are also improved through logical speedup, 
achieved with the help of link grouping. 
STATE OF THE ART. 

Methods available thus far for offering a wideband link 

3 5 are : 

1. high physical speed, 

2. series-parallel conversion in order to attain higher 



wo 93/19550 



PCT/SE93/ee227 



2 

logical speed than the physical one. 

Disadvantages with the first method are the difficulty in 
realizing high speed electronics and mechanics, as well as the 
consequent power dissipation. The other method builds normally 
5 on multiplexing at bit-level and therefore requires 
bit-synchronous transmission. 

Among publications discussing the need to create greater 
bandwidth them that which is physically available can be 
mentioned EP 0 374 574, WO 85/04300, WO 90/12467, and IBM 
10 Technical Disclosure Bulletin Vol 32, No. 9A, February 1990, 
pp. 45-49. 



STATEMENT OF THE TWVENTIQW. 

One object with the present invention is to provide a 
15 method for doing link grouping of the type presented by way of 
introduction, which offers a simpler solution than previously 
known for the problem in the ATM context of achieving higher 
bandwidth than that which is physically available. 

The method according to the invention comprises the steps 

20 of: 

grouping r according to a link protocol, a plurality of 
incoming physical links into a link group, wherein the incoming 
physical links are connected in parallel to respective input 
ports, the link group has a bandwidth that is a sum of 

25 bandwidths of the grouped incoming physical links, and the link 
protocol requires labels in packets communicated via the link 
group, the labels describing a route through the switch that 
holds together the grouped incoming physical links and each 
label comprising at least one bit that describes the route and 

30 is identical to corresponding bits in labels of other packets 
communicated via the link group; and 

distributing packets communicated via the grouped incoming 
physical links among a plurality of outgoing physical links 
connected in parallel to output ports of the switch. 

35 

DESCRIPTTON OF DRAWINGS. 

The invention shall now be described in more detail with 
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reference to the attached drawings, on which 

Figure i schematically illustrates the general principle of 
link grouping to create logical links in an ATM switch. 
Figure 2 shows a protocol model illustrating how to 
provide, according to the invention, link grouping in switches, 
Figure 3 schematically illustrates multi-port link grouping 
according to the invention. 

Figure 4 similarly schematically illustrates multi-port 
link grouping in a shared buffer pool 

Figure 5 similarly schematically illustrates transmission 
of an internal flow of packets in one block to another block 
via grouped links. 

Figure 6 show a logical structure for link grouping over a 
link between nodes, 
15 Figure 7 schematically illustrates label attachment in 

switch ports for interport link grouping according to the 
invention. 
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PREFERR ED EMBODIMENTS. 

The same reference numbers are used in the various Figures 
for the same or equivalent details. 

In fig. 1, label 2 indicates a switch core in a packet 
switch for packets with constant length, e.g. an ATM switch. A 
number of parallel physical links 4 arrive at input switch 
25 ports 6 to the core 2. Before the core 2 occurs a link 

grouping of the physical links 4 for forming logical links, 
each consisting of several of the parallel links 4. This link 
grouping is indicated in fig. i, as in the following Figures, 
with ovals, for example, the ovals 1 and m in fig. i. The link 
30 grouping at the core's 2 outputs need not necessarily be the 
same as at the inputs, which is indicated in fig. 1 where link 
groupings n and r at the outputs correspond to link groupings 1 
and m, respectively, at the inputs. 

Via output switch ports 8 a number of parallel, physical 
35 links 10 then leave the switch. 

The link grouping is characterized more particularly in 
that the individual links in a group carry serial packet flows 
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at the the interface of the switch core but constitute one 
logical link by use of protocols (supported by hardware) which 
are specific for a certain grouping definition. The link 
grouping should be as general as possible, that is, grouping in 
5 many different group sizes should be possible, as shown in fig. 
1. However, it is in practice most interesting to have the 
same link grouping at inputs and outputs, and to have groups 
which are multiples of four physical links. 

The link grouping can be described using a protocol stack 
10 of the type shown in fig. 2. The upper part of this Figure 
indicates schematically how the switch core 2 can be divided 
into a number of nodes 12 and transmission links 14, where the 
nodes 12 perform space selection and the transmission links 14 
offer point-to-point transmission between the nodes. 
15 Between the switch ports 6 and the nodes 12 the link 

grouping is defined by a protocol on level 2, G2, and over the 
transmission links a protocol on level 1, Gl, is used. The 
association between elements in the upper and lower parts of 
fig. 2 is indicated by double arrows. 
20 That which is common for all thinkable variations of link 

grouping according to the invention is that the label number in 
the head of the packets describes a route that holds together 
the grouped links through the entire switch, which means that 
the bits in the label which describe the route over a certain 
25 transmission link are the same for packets which belong to the 
same group. This information is a part of G2. 

Depending on how the rest of the protocol looks in level 2 
{G2), link grouping can be divided into two types, namely 
multiport and interport link grouping. 
30 Multiport link grouping shall be discussed first here with 

reference to Figures 3 to 6, 

The advantage to this type of link grouping is that no 
eactra fimctions are required in the switch ports, and its 
mechanism can more particularly be divided into what happens 
35 partly in the nodes and partly over the transmission links. 

m a node the group is held together by using similar space 
selection for the grouped links, for example with the help of 
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certain bits in the label number being identical, so that they 
come out on the same grouped output and the sequence is 
retained. This can be realized, for example, according to what 
is shown in fig. 3, by links belonging to the same group, or 
5 logical links, for example n or m, being written into the same 
buffer 16 and 18, respectively, in the node 12, and being read 
out from the node serially on out-going links. 

The common buffer principle, indicated with 20 in fig. 4, 
can otherwise also be used, wherein link grouping and readout 
10 can be controlled by a function 22 for memory handling 

according to known principles, see for example IEEE JOURNAL ON 
SELECTED AREAS IN COMMUNICATIONS, October 1991, volume 9, 
number 8, p. 1239, "32*32 Shared Buffer Type ATM Switch VLSI's 
for B-ISDN's". In fig. 4 out-going logical links have been 
15 labeled with 24 and 26. 

The functionality which must be introduced over a link is 
some type of simple link protocol (Gl) in order to send and 
re-create a packet flow over a "parallel connection- with a 
division on the packet level according to the description 
20 below. 

With reference to fig. 5, a flow 28 of packets 3 0 with 
constant length internally in a block can be transmitted to 
another block via external links 32. The internal flow can be 
said to have a packet frequency of F packets/s. It follows 
25 that the time for a packet in the internal flow can be said to 
be t-l/F. 

The external links 32 can because of technical limits have 
a transmission capacity which is limited such that an external 
link can not handle a packet frequency of f, but only lower 
30 packet frequencies. in such a case can several external links 
32 together, through grouping, transmit the internal packet 
flow 28. 

When n external links are grouped and the packets 3 0 in the 
internal flow 28 are divided among them the packet frequency on 
35 every external link becomes F/n. It follows that the time for 
every packet on the external links becomes nt. 

The sequence order between the packets is important and 
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must be preserved- It is therefore important that the packets 
30 are transmitted on the grouped links 32 in such a way that 
the sequence ordering can be preserved • In order for this to be 
possible grouping logic 34 is needed on the sending side, and 
5 restoration logic 36 on the receiving side. 

The send and receive logic, 34 and 3 6 respectively, should 
be designed so that a certain spread in transmission time is 
allowed between the external links 32. 

The link protocol includes firstly an algorithm for how the 
10 packets shall be portioned out and gathered in between the 

physical links ^ such that space correctness is preserved, for 
example in the sequence 1, 2, 3, 4. 

Secondly, the protocol includes synchronization which 
secures that the packets are transmitted such that there exists 
15 a determined time pattern between the physical links- For 
example the packets can be sent out simultaneously or time 
delayed by a constant interval. The time delay/difference in 
length on the links, can be allowed to correspond to one packet 
if suitable bit and packet synchronization is introduced, 
20 First a short description shall be given here of a suitable 

design for the sending and receiving logic. 

The design should be done so that the packets 30 in the 
internal flow 28 are divided over the external links 32 
according to a given pattern. One suitable pattern is a 
25 cyclical one according to the following: Packet 1 is sent on 

external link 32,1, packet 2 on link 32.2, etc., until packet n 
on link 32 .n; thereafter packet n+1 on link 32,1, etc., 
according to fig. 5- The packets on the external links should 
be sent with a mutual time delay such that if a packet (the 
30 begin of packets) is sent at time 0, packet 2 is sent at time 
t, etc., until packet n is sent at time {n-l)t, and thereafter 
packet n+1 at time nt. 

Since the division is done at packet level, two buffer 
stages, 38 and 39, must be introduced according to fig* 6 
35 because the bandwidth in to and out of the send queue is higher 
than the bandwidth of the physical links. The buffer stages 
compensate for the difference in bandwidth. 
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The grouping logic is built consisting of a distributing 
unit 42, one of the buffers 38.1 to 38. n per external link in 
the buffer stage 38, and a sequence control unit 44 that 
controls the time ordering between the packets on the external 
5 links. The distributing unit 42 allocates the packets, packet 
for packet, to the buffers 38.1 to 38. n toward the external 
links such that the distribution of packets between the links 
is obtained as described above. 

The buffers 38.x to 38. n, one to every external link, 
10 handles the differences in time between transmission of 1 

packet on the internal flow verses on the external links, a 
buffer is filled with a packet from the internal flow in the 
time t. m the next time period of nt it is emptied toward the 
external link. To secure that the packet sequence is maintained 
15 and no packets pass by each other in the buffers, every buffer 
should have room for more than l-i/n packets and less than 
2-l/n packets. 

The unit 44 for controlling the time ordering sees to it 
that the above indicated time ordering is maintained. The unit 

20 uses the packet flow on external link 32.1 as reference and 
controls the flows out of the buffers 32.2 to 32. n toward the 
other links based on this reference. Since the packet frequency 
IS the same for all the external links the unit only needs to 
be activated upon initial startup or when some external link is 

25 restarted. 

The restoration logic 36 consists of one of the buffers 
40.1 to 40.n per external link, a gathering unit 46, and a 
sequence restoration unit 48. 

Differences in transmission times between the different 
external links can arise. These differences can arise in the 
grouping logic 34, in the transport over the external links, as 
well as in the restoration logic 36. The restoration logic 
must therefore be built such that it can handle a spread in 
arrival time for the packets on the external links. 

External link 32.1 is used as the reference. on the other 
external links the nominal arrival times for the packets are 
calculated starting from the reference. From the nominal 
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arrival time a spread of +d is allowed. Normally d can be 
allowed to be d=nt/2, i.e-, a half packet time on the grouped 
links. 

The buffers 40 to the external links 32 are used to handle 
5 the differences in packet time between the external links and 
the internal flow. a buffer is filled with a packet from the 
external link during the time nt. The buffer is emptied 
thereafter toward the internal flow in the time t. In the 
buffers 40 the packet is also stored for an extra time of 
10 nominally d and maximum 2d to handle differences in actual 

verses nominal arrival time for packets on different links. The 
buffer needs to be maximiim 2 packets long. 

The gathering unit 46 picks up packets from the buffers 40 
to the external links according to the order that is described 
15 above . 

The sequence restoring unit 48 sees to it that packets are 
picked up in the order that is described above. 

With interport link grouping the protocols act only between 
the switch ports. Two methods are conceivable here. 

20 The first method is label based. With reference to fig. 7, 

a label, based on VCI/VPI (Virtual Circuit Identifier/ Virtual 
Path Identifier) information, is attached in the switch ports 6 
to arriving packets in sequence for the grouped links. The 
switch core does not need to "know" anything about which 

25 physical links belong together, only level 2 protocol is 
needed. The packets are supplied suitably with sequence 
numbers to be able to restore the sequence order if packets 
within the same group are delayed by different amounts of time 
along different physical routes. 

30 With the above mentioned label attachment in the switch 

ports 6 a buffer is used that has a number of outputs 
corresponding to the number of links to be grouped. The links 
are written into the buffer and are read out sequentially on 
the outputs. The read-out routine is preferably arranged so 

35 that the incoming packets are distributed evenly over all the 
links in the group. The grouped packets also receive a label 
code that essentially contains the same bits. The grouping 
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means must be able to buffer enough packets to be able, with 
help of the labels, to choose packets that shall go out on the 
same logical link, correct the sequence, and evenly distribute 
them on all the links in the group. odd numbers of packets are 
filled out with empty packets. The link terminator 8 also 
contains a buffer large enough to enable correction of the 
sequence . 

The other method is based on packet synchronism through the 
switch core. Protocols on both level i and level 2 are used. 

The protocol on level 2 is based on a label attacher and a 
link terminator, as in method one. The difference is that no 
sequence number is needed to secure the sequence, nor is any 
large buffer needed. On level 1 packet synchronism is secured 
in and between the switch ports and the nodes, so that the 
15 packets in the group move through the entire switch 

packet-synchronously. Sorting in the switch ports is carried 
through with help of the bits in the label that are different 
between packets within the group. 
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CLAIMS . 

1. A method of grouping links in a packet switch for 
constant-length packets, the packet switch having a plurality 
of nodes and a plurality of transmission links for selectively 
5 connecting a plurality of input ports to a plurality of output 
ports via routes through the switch, wherein the nodes enable 
space selection and the transmission links interconnect the 
nodes, comprising the steps ofs 

grouping, according to a link protocol, a plurality of 
10 incoming physical links into a link group, wherein the incoming 
physical links are connected in parallel to respective input 
ports, the link group has a bandwidth that is a sum of 
bandwidths of the grouped incoming physical links, and the link 
protocol requires labels in packets communicated via the link 
15 group, the labels describing a route through the switch that 
holds together the grouped incoming physical links and each 
label comprising at least one bit that describes the route and 
is identical to corresponding bits in labels of other packets 
communicated via the link group? and 

distributing packets communicated via the grouped incoming 
physical links among a plurality of outgoing physical links 
connected in parallel to output ports of the switch. 

2. The method of claim i, wherein the link group is held 
together by space selection in the nodes based on the labels 
such that a sequence of the grouped incoming physical links are 
connected to a corresponding sequence of outgoing physical 
links. 

3. The method of claim 2, wherein each node includes a 
plurality of buffer memories, the grouping step includes the 
step of storing packets communicated via the link group in the 
same buffer memory in each node, and the distributing step 
includes the steps of retrieving the stored packets from the 
buffer memories and providing the retrieved packets to the 
outcroing physical links. 

35 4. The method of claim 2, wherein each node includes a 

buffer memory, the grouping step includes the step of storing 
packets communicated via a plurality of link groups in the 
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buffer memory in each node, and the distributing step includes 
the steps of retrieving the stored packets from the buffer 
memories and providing the retrieved packets to respective 
groups of outgoing physical links. 

5. The method of any of claims 2 to 4 , wherein the link 
protocol provides for properly sequencing packets communicated 
via the link group and synchronizing such packets based on a 
predetermined timing pattern. 

6. The method of claim 5, wherein the link protocol 
provides for one of simultaneously transmitting the packets 
communicated via the link group and transmitting the packets 
communicated via the link group with a predetermined time delay 
between such packets. 

7. The method of claim 1, wherein the link protocol acts on 
packets only between the input ports and output ports, 
providing for interport link grouping. 

8. The method of claim 7, further comprising the steps of 
sequentially adding labels to packets arriving at the switch 
via the link group and sorting such packets based on the labels 
so that an arrival sequence of such packets is preserved. 

9. The method of claim 1, wherein the switch includes a 
buffer memory having a plurality of outputs corresponding to 
the plurality of incoming physical links in the link group, the 
grouping step includes the step of storing packets communicated 
vaa the link group in the buffer memory, and the distributing 
step includes the steps of sequentially retrieving the stored 
packets from the buffer memory and providing the retrieved 
packets to the buffer's outputs - 

10. The method of claim 9, wherein the retrieved packets 
are provided evenly to the buffer's outputs. 

11. The method of claim 10, wherein the labels in packets 
communicated via the link group each comprise a predetermined 
set of bits. 

12. The method of claim 11, wherein a plurality of packets 
are stored in the buffer memory, the distributing step includes 
the steps of selecting packets to be retrieved for an outgoing 
link group comprising a plurality of outgoing physical links. 
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and the retrieved packets are provided evenly to the plurality 
of outgoing physical links. 

13, The method of claim 12, wherein the distributing step 
includes the step of padding a plurality of packets comprising 

5 an odd number of packets with a predetermined number of empty 
packets • 

14, The method of any of claims 9 to 13 , including the step 
of synchronizing the packets during the grouping and 
distributing steps such that the packets flow synchronously 

10 through the switch. 

15, The method of claim 14, wherein the grouping step 
includes the step of sorting the packets based on differences 
between bits in the packets' labels • 
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